Sport program chaptering

ABSTRACT

In certain implementations, a method consistent with the invention involves at a television receiver device, receiving a television program depicting a sports program, where data associated with the sports program is represented in a score graphic that is displayed on a television display overlaying images of the sports program; storing the television program depicting the sports program; analyzing the score graphic to identify a change in at least one element of data displayed in the score graphic from one video frame to a subsequent video frame; and storing a chapter mark associated with the time that the change in the element of data displayed in the score graphic occurred. This abstract is not to be considered limiting, since other embodiments may deviate from the features described in this abstract.

COPYRIGHT NOTICE

A portion of the disclosure of this patent document contains materialwhich is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction of the patent document or thepatent disclosure, as it appears in the Patent and Trademark Officepatent file or records, but otherwise reserves all copyright rightswhatsoever.

BACKGROUND

Personal Video Recorders can be used to record sports programs and thelike for later viewing. A user can manually scan through the recordingfor highlights in order to view the sports program in a shorter amountof time.

Video production of sports programs and the like often contain on-screengraphics lasting substantially the full duration of the event, to updatethe viewer on important statistics such as the score and time remaining.

BRIEF DESCRIPTION OF THE DRAWINGS

Certain illustrative embodiments illustrating organization and method ofoperation, together with objects and advantages may be best understoodby reference to the detailed description that follows taken inconjunction with the accompanying drawings in which:

FIG. 1 is a drawing depicting a television display with a “scoregraphic” as is used in implementations consistent with certainembodiments of the present invention.

FIG. 2 is another drawing of the television display after a touchdownscore is posted to the score graphic as is used in a manner consistentwith certain embodiments of the present invention.

FIG. 3 is another drawing of the television display after apoint-after-touchdown score is posted to the score graphic as is used ina manner consistent with certain embodiments of the present invention.

FIG. 4 is an example of a flow chart of an illustrative chapter markingprocess that operates in a manner consistent with certain embodiments ofthe present invention.

FIG. 5, which is made up of FIGS. 5A-5F is an example of a baseballscore graphic that chronicles an illustrative succession of events in ahypothetical baseball game in a manner consistent with certainembodiments of the present invention.

FIG. 6 is an example playback window for selection of events that are tobe retrieved using chapter marks made in a manner consistent withcertain embodiments of the present invention.

FIG. 7 is an example of an illustrative process for playback using thechapter marks established in a manner consistent with certainembodiments of the present invention.

FIG. 8 is a simplified filtering system for filtering results of changesin a baseball score graphic in a manner consistent with certainembodiments of the present invention.

FIG. 9 is an illustrative time line for score graphic events and chaptermarks consistent with certain embodiments of the present invention.

FIG. 10 is an illustrative time line for score graphic events thattrigger playback at a time interval offset in a manner consistent withcertain embodiments of the present invention.

FIG. 11 is an example block diagram of a television receiver device thatoperates in a manner consistent with certain embodiments of the presentinvention.

DETAILED DESCRIPTION

While this invention is susceptible of embodiment in many differentforms, there is shown in the drawings and will herein be described indetail specific embodiments, with the understanding that the presentdisclosure of such embodiments is to be considered as an example of theprinciples and not intended to limit the invention to the specificembodiments shown and described. In the description below, likereference numerals are used to describe the same, similar orcorresponding parts in the several views of the drawings.

The terms “a” or “an”, as used herein, are defined as one or more thanone. The term “plurality”, as used herein, is defined as two or morethan two. The term “another”, as used herein, is defined as at least asecond or more. The terms “including” and/or “having”, as used herein,are defined as comprising (i.e., open language). The term “coupled”, asused herein, is defined as connected, although not necessarily directly,and not necessarily mechanically. The term “program” or “computerprogram” or similar terms, as used herein, is defined as a sequence ofinstructions designed for execution on a computer system. A “program”,or “computer program”, may include a subroutine, a function, aprocedure, an object method, an object implementation, in an executableapplication, an applet, a servlet, a source code, an object code, ascript, a program module, a shared library/dynamic load library and/orother sequence of instructions designed for execution on a computersystem.

The term “program”, as used herein, may also be used in a second context(the above definition being for the first context). In the secondcontext, the term is used in the sense of a “television program”. Inthis context, the term is used to mean any coherent sequence of audiovideo content such as those which would be interpreted as and reportedin an electronic program guide (EPG) as a single television program,without regard for whether the content is a movie, sports program,segment of a multi-part series, news broadcast, etc. In particular, theterm as used in this application relates particularly well to “sportsprograms”, where the term “sport” is intended to be broadly interpretedto encompass any competitive program that has a score graphic (whichmight also include competitive games such as poker or other gameprograms that incorporate such a graphic, despite there not being a truesport or athletic component to the competition).

Reference throughout this document to “one embodiment”, “certainembodiments”, “an embodiment”, “an implementation”, “an example” orsimilar terms means that a particular feature, structure, orcharacteristic described in connection with the embodiment is includedin at least one embodiment of the present invention. Thus, theappearances of such phrases or in various places throughout thisspecification are not necessarily all referring to the same embodiment.Furthermore, the particular features, structures, or characteristics maybe combined in any suitable manner in one or more embodiments withoutlimitation.

The term “or” as used herein is to be interpreted as an inclusive ormeaning any one or any combination. Therefore, “A, B or C” means “any ofthe following: A; B; C; A and B; A and C; B and C; A, B and C”. Anexception to this definition will occur only when a combination ofelements, functions, steps or acts are in some way inherently mutuallyexclusive.

The user of a digital television (DTV) receiver with personal videorecorder PVR (sometimes referred to as a digital video recorder or DVR)functionality (either internal or external) may set it up to record asports program such as a baseball, football, or basketball game. Often,when the user comes back later to view the recording, they do not havetime to view the entire broadcast and would like to skip ahead tovarious “highlight” moments. Using the example of a baseball game, theuser could choose to review just a) scoring plays; b) plays resulting inone or more outs; c) plays resulting in base runners moving; d) playsresulting in home runs; e) home runs; f) double plays, or any logicalcombination of those, among other possibilities. With a football game,one may wish to only view scoring plays, changes of possession, etc.With today's PVR functionality, the user may manually fast-forward toareas of interest, but if distracted may miss a desired scoring play orthe like. Furthermore, a measure of close attention and effort isrequired during the fast-forwarding to position the playback at allpoints of interest within the game.

In accord with implementations consistent with embodiments of thepresent invention, an automated method for deriving chapter pointsassociated with the highlights can be provided. It has been observedthat such chapter points can often be deduced from observation of theon-screen graphical images that form a part of a television presentationof a sports program. A “score graphic” is a video overlay provided by abroadcaster that gives the viewer an ongoing snapshot of current dataabout a game in progress such as the score, timing, and other gamestatus information. The video images can be sampled from the broadcastand used to set chapter points corresponding to areas of the game havinga high probability of being of interest due to changes identified in thescore graphic. Such changes in the score graphic can be referred to as“score graphic events” or simply “events” or the like. The term “eventrecords” can be used herein to denote the information associated with ascore graphic event, such as the timing of the event, what element(s)changed, and the to/from values of the elements. These event records canbe saved at the receiver in any suitable format. The points in playbacktime are marked using “chapter marks”, which can be implemented in amanner similar to that used for marking other chapter points in anaudio/video program. “Chapter marks” can be used herein to denote marksor stored times that are associated with changes (events) in a scoregraphic that are likely to represent points of playback time wheresignificant points of interest occur during the course of a sportingprogram. Such chapter marks may be stored and conveyed in any suitablemanner for use by the viewer in retrieval of content starting at or nearthe chapter point as described herein.

With reference to FIGS. 1-3, an example of a score graphic used in afootball game is depicted on a television display 100. In this example,referring first to FIG. 1, a simple score graphic 104 is depicted inwhich the two teams playing American style football are identified onthe left side as PTS and GBP in blocks 108 and 112. Each team's score isshown below their identifiers in blocks 116 and 120. The quarter andtime remaining in the quarter appears in block 124. It will beunderstood that the score graphic appears as an overlay over the imagesfrom the actual game as it is being played, and the overlay is createdby the network in producing the television show of the game and is henceencoded, for example, within the broadcast video image (as opposed tobeing formed by accompanying metadata that is rendered at the TV set).

The score graphic generally appears in a consistent location on thedisplay frame and can therefore be readily identified either in advanceby knowledge of the way such score graphics are presented by eachnetwork, or by searching for an area of the display that is relativelystatic or any other suitable technique. Hence, once the score graphic isidentified and located on the display, it can be analyzed (e.g., bypattern matching and/or character recognition technology) for asubstantial amount of information that is directly related to many gamehighlights.

By way of example, in FIG. 1, the score is shown to be 7 to 3 in favorof GBP. When the score graphic display changes in certain areas, it isapparent that a significant play has occurred in the football game. Itis noted that some changes are generally insignificant such as secondsticking off of a clock (usually with possible exceptions such as finalseconds of a tied game, which may be a significant play but could alsobe represented by detection of a time of 0 on the clock).

In this example case, consider FIG. 2 in which the score has changed inbox 120 to 13 to 3 in favor of GBP. If the information in box 120 ismonitored by the television system, it is readily deduced that justprior to the score graphic change, a significant play has taken place.In this case, a touchdown has been scored by GBP. Hence, if therecording is chapter-marked either at the point of the score change orsome reasonable time in advance of the score change, the viewer caneasily move to that point to view the play by simply activating a remotecontrol instruction to jump to the next chapter mark. In the case offootball, the viewer may wish to advance to approximately one realminute (for example, versus game minute) prior to the score change. Theviewer can then view the scoring play which is a likely candidate for ahighlight.

Similarly, FIG. 3 shows the score again changing at 120 by a singlepoint, which suggests a successful point-after-touchdown kick. With thissimple example in mind, one can readily deduce how an event orcombination of events can lead to a deduction that play of interest hasjust occurred in the recorded game that has high potential to be ahighlight moment. It will also become evident upon consideration of thepresent teachings that certain games are very amenable to analysis inthis manner to produce a high likelihood of creating chapter points thatare close to game highlight moments. It is likely, for example, thatgames with lower numbers of scoring plays such as baseball, football,soccer, and hockey are very likely to be conducive to successful andefficient analysis (producing a sequence of highlights that can compressa game of several hours into several minutes) whereas games with manyscoring plays such as basketball may not be as productively analyzed inthis manner. That notwithstanding, most programs having a score graphiccan be analyzed using techniques consistent with the present teachings.

It will also become evident upon consideration of the present teachingsthat different plays in different sports may be conducive to differentamounts of viewing time prior to a score graphic event (a change in anyfeature of the score graphic display). It will become equally evidentupon consideration of the present teachings that more events tracked inthe score graphic can lead to a more detailed analysis of events thatcan be logically combined to deduce that a particular event has occurredthat can be marked to delineate chapter points for subsequent playback.

In any case, the basic process that is used with implementationsconsistent with the present invention is depicted in FIG. 4 as process400 starting at 402. At 406, any suitable process is used to identifyand isolate the location of a score graphic in the presentation on thedisplay. This can be accomplished using a number of techniques includingreferencing a catalog of data regarding such score graphics. For examplea television receiver device (TV, Set-top Box, PVR, etc.) may be able toreadily determine what channel is being tuned, which network the channelis affiliated with and what game is being presented from metadata usedfor purposes of a generation of an electronic program guide (EPG). Or, auser can explicitly identify the score graphic in some manner. Or, thedisplay can be scanned for information that identifies a segment of thescreen as being dedicated to presentation of the score graphic. Videosamples taken over a period of time can quickly identify rectangularareas in which few if any frame-to-frame changes occur (e.g. staticgraphics). In any case, once the score graphic is identified in thedisplay, it generally remains in place for most of the game except forcommercial breaks, commentary breaks, half-time, time-outs and othersuch occurrences. The score graphic will appear with frequency on thedisplay in a relatively fixed location and have a particular set ofinformation that is always presented. In the prior example, the scoregraphic is very simple and presents only the teams, score, time andquarter. However, more information is often presented in score graphicsas will be discussed later.

Once the score graphic is identified and located, the area of the frameincluding the score graphic image is captured at 410 (which may simplyinvolve reading a portion of the information stored in a full-framevideo buffer). This frame image is then processed at 414 using an OCRengine and/or pattern analysis or pattern matching algorithms to extractthe available data from the score graphic. If the analysis determines at418 that the score graphic is missing, then the situation may be onlytemporary and due to a commercial break or a change in picture tocommentators or other temporary situation. Assuming it is determined at422 that the game or recording is not actually ended, the process waitsfor a reasonable period of time (perhaps 1-5 seconds) at 426 and a newframe is captured at 410 and analyzed at 414.

When the score graphic has returned or is not deemed missing at 418, theprocess analyzes the score graphic to see if any relevant informationhas changed (other than perhaps only game time information) at 430.Whenever a significant change takes place (where significance can bedetermined by a change that could potentially represent a score orpossible highlight play, e.g., if a significant number of replays ispresented by the network—this may cause the score graphic time toactually change direction as the replay is presented) then the time ofthe change in the score graphic, as well as other pertinent data(exactly what changed, and the to/from values) are recorded as a part ofthe event record at 434. Via a separate process, the timing and types ofchanges in the score graphic can be filtered and converted to chapterpoints to permit the viewer to quickly review highlights of the game byuse of a chapter advance remote control button for example, so as to beable to skip less interesting portions of the game.

The analysis of the score graphic events proceeds in this manner everytime interval (e.g., every several seconds) until the game ends at 422at which point the process ends at 440. At this point, the viewer has astored copy of the entire game (or whatever time span was designated forrecording) which can be either viewed in its entirety or viewed in anabbreviated manner by jumping from highlight to highlight in a mannerwhich will be described later.

The event records created using the above process can be stored in anysuitable manner and format that can be readily retrieved during aplayback process in order to facilitate the user skipping from onehighlight to the next, where the event records can include framenumbers, or playback times or any other convenient data representation.It is noted that the chapter marking process described above may becarried out as the sports program is being presented (in near real time)or as a post storage processing action without limitation.

FIGS. 5A-5F, show an example score graphic that might be used along withpresentation of a baseball game. To simplify the illustration, thedisplay itself is omitted, but it will be understood that the image ofthe score graphic will typically overlay a portion of the display suchas an area near an upper or lower corner. The score graphic 500indicates information about a baseball game including:

-   -   The teams playing at 504 and 508;    -   The current score (number of runs scored thus far by each team)        at 512 and 516;    -   The inning at 520;    -   An indicator of whether it is the top or bottom of that inning        at 524;    -   A number of outs in the inning at 528;    -   A pitch count (balls and strikes) also in box 528; and    -   A number and location of base runners at 532 as indicated        graphically with a representation of a baseball diamond similar        to that used in a scorer's book.

In this example of FIG. 5A, the score is 0 to 0 in the bottom (lasthalf) of the third inning, the pitch count is one ball and no strikes,there is one out and a runner is currently on first base.

Straightforward analysis of the video samples from the recorded game cantake note of any time in which the score graphic changes in certainways, to record an event record where there is a:

Change in score;

Change in base runners;

Change in number of outs;

Change in inning (or which half of the inning); and

Change in pitch count.

At some points, changes in multiple things will occur. When thishappens, or in some cases when certain isolated events happen, one candeduce not only that a significant play has taken place, but what typeof play has taken place. This permits the viewer to pick and choose thetypes of plays he or she wishes to view as a highlight by use ofselective filtering of the changes in data presented in the scoregraphic. Thus, certain types of plays can be determined by understandingthe meaning of certain changes. For example:

-   -   A home run is indicated by a change in score corresponding to        one run plus the number of base runners that were on base at the        beginning of the at-bat;    -   A base hit or walk is indicated by a change in base runner        position where first base is occupied at the end of the play and        no outs were recorded;    -   A strikeout can be deduced if the number of outs changes or an        inning (or half of an inning changes) after a count of two        strikes;    -   Some types of stolen bases can be deduced, for example if runner        advances from one base score graphic sample to another;    -   A grand slam occurs when the score advances by four points in        one jump; and    -   Etc.

A full decision tree that provides for logical interpretation of singlechanges and combinations of changes can be readily devised by one whounderstands the game being played and the representation of plays in thescore graphic. In the simplest of implementations, chapter marks cansimply signify playback times for changes in score as in the footballexample of earlier. For more complex score graphics, combinations ofscore graphic events can be used to deduce that certain actions have orare likely to have occurred. However, it is observed that in someinstances a change in an element of the score graphic can be ambiguous.Nevertheless, even ambiguities are not particularly troublesome ingeneral since the derived information is only used to create a chaptermark and the viewer can deduce what has actually occurred by viewing theactual play and listening to the associated commentary. When theanalysis deduces the type of play, this information can also be used todetermine how much earlier the play of interest is likely to havestarted.

For illustrative purposes, consider the following sequence of plays in abaseball game depicted by the score graphic progression of FIG. 5. Inthe example shown, it can be readily deduced that when the score graphicchanges directly from that of FIG. 5A to that of FIG. 5B that a playoccurred in which either the batter or the runner at first base was putout since there is still a runner at first base shown in 532. Since thischange took place with the pitch count at 1-0 at 528 and the number ofouts advanced, it is clear that the batter hit the ball and the out cameas a result. Hence, this may be a play that the viewer wishes to see ifhe or she wishes to view all field plays resulting in an out.

It is noted that as mentioned above some changes in a score graphicstatus can represent several possible plays. So for example, in the caseof the transition leading to FIG. 5A, if a pitch count was 3 balls, 2strikes and there is one out, and a transition is to no pitch count anda runner at first, the score graphic transition could represent at leastsix scenarios: 1) the batter was walked on four balls, 2) the batter gota base hit, 3) the batter reached first base on an error after a hitball, 4) the batter was hit by a pitched ball, 5) the batter reachedfirst on a balk, or 6) the batter stole first base. Unless the scoregraphic conveys more information (e.g., is more informative via a dropdown window or the like, or is more complex than the score graphicshown) it is impossible to deduce which of these possibilities occurred.However, if a viewer is interested in changes in status of runners onbase, such occurrence will be noted by deriving a chapter mark and theviewer still will have the opportunity to determine what exactlyhappened by viewing the actual playback.

Continuing this example to transition to FIG. 5C, it is noted that therunner at 532 has advanced to second base, the count has advanced butthere is no change in outs at 528 and no inning or half inning change at520 or 524. Hence, the likely scenario in this case is that the runnerat first stole second base—possibly after a wild pitch (since there aretwo outs). Hence, this may be a play that the viewer wishes to view ifhe or she wants to see all advances in base runners.

Continuing this example to a transition to FIG. 5D, it is noted that therunner at second base is shown at 532 to have advanced to third base,the count has disappeared and there is no change in outs at 528 and noinning or half inning change at 520 or 524. Hence, the likely scenarioin this case is that the batter hit the ball and advanced to first baseand the runner at second base advanced to third base. Hence, this mayalso be a play that the viewer wishes to view if he or she wants to seeall advances in base runners or all hits (although an error might haveoccurred which is not shown in this score graphic).

Continuing further with this example to the transition to FIG. 5E, it isnoted that the runners at first and third bases are shown at 532 to havedisappeared, there is no count shown and there is no change in outs at528 and no inning or half inning change at 520 or 524. But, the scorehas advanced by three. Hence, the likely scenario in this case is thatthe batter hit a home run advancing himself and the two base runners toscore. Hence, this may also be a play that the viewer wishes to view ifhe or she wants to see all advances in base runners or all hits(although an error might have occurred which is not shown in this scoregraphic) or all home runs.

Finally continuing this example to the transition to FIG. 5F, it isnoted that there are no runners on base at 532, the first pitch of theinning apparently has not been thrown as indicated further by 528 andthere are no outs. However, the inning has changed to the top of thefourth inning at 524 and 520. The score has not changed, so it isapparent that the third inning has ended. This potentially representsseveral types of plays that may be of interest—change of inning (andpossible change of pitcher from last inning) and an out (possibly astrikeout if the last count showed two strikes. Hence, the likelyscenario in this case is that the batter was out on the first pitch,ending the third inning. Hence, this may also be a play that the viewerwishes to view if he or she wants to see all plays resulting in outs.

So, for example, if the user wishes to review only scoring plays thosepoints in the program just prior to changes in the displayed scores canbe filtered out and used as chapter points. “Just preceding” can bedetermined as a general rule, allowing for possible user adjustment andrefinement after further experience with processing multiple games, butmay be set to a fixed value of approximately 20 seconds for many games.This value can be refined by experimentation or established by userselection. If the user wishes to review all plays which resulted inscores or outs, again, that can be determined by taking note of changesin the score graphic. The chapter marks can be set at either the time ofdetection of the score graphic event, and then the video playback pointbacked up in time to a point just prior to that point, or the mark canbe placed at the point where the user will actually advance.

With reference to FIG. 6, menu 600 illustrates an example user interfacewindow wherein the user is asked how they would like to review aparticular recorded baseball game. Chapter points are filtered andactivated based on the selected options. The chapter points can beanalyzed and categorized either at the time of marking the chapterpoints or at the time of playback or offline without limitation. In thisexample, the user can select any or all of the following types of playswhich are deduced from analysis of changes in the score graphic:

Plays resulting in out(s);

Scoring plays;

Home runs;

Moving base runners;

Base hits/walks; and

Next inning/half-inning.

In the illustrated example, the user has utilized a remote control in aconventional manner or other user interface to select plays resulting inouts at 604, home runs at 608 and base hits/walks at 612. The viewingstarts when the user selects “start” button 620 from the menu using aremote controller.

While analysis of a baseball game and a football game are brieflydescribed herein, the technique described may be applied to other sportssuch as tennis, basketball, soccer, hockey, etc., or other games such aspoker and contests, etc., with each sport having a particular algorithmapplied to analysis of the changes to the score graphic. Moreover, bothdigital television sets and PVR devices or STBs having PVR functions(each of which has PVR functionality) can be utilized to realizeimplementations consistent with certain embodiments consistent with thepresent invention without limitation.

An example of a playback method utilizing the menu 600 is depicted inFIG. 7 as process 700 starting at 704 after which the viewer elects toplay back a recorded program containing a sports program in general or abaseball game in this particular example at 708. The viewer is presentedwith a menu 600 which provides for selections of several types of playsthat the viewer wishes to see in his or her playback at 712. Once allselections are made at 714, the viewer initiates the start of theviewing experience from the menu or via any equivalent command and theprocess 700 seeks the first (the next is first in this case) chaptermark at 716. In other words, on user command, the playback “jumps” oradvances quickly to the time associated with the chapter mark. Theprocess continues to 720 where the changes in the score graphic areanalyzed or filtered to determine if a match has been found to thecriteria established by the viewer in menu 600. If so, playback beginsat 724 at a point in time just prior to the change event that took placein the score graphic. This process will be discussed further at a laterpoint in this document.

Once playback begins, in this implementation, it continues until theviewer issues a command from the remote controller or other suitableinterface that indicates that he or she wishes to proceed to the nextchapter mark at 728. Assuming the prior chapter mark was not the lastchapter mark as determined at 732, this election by the viewer resultsin the process returning to 716 where the next chapter mark is found andthe process repeats. Until such time as the viewer elects to move to thenext chapter mark, playback continues at 736 until the recording (game)ends at 740 thereby ending the process at 750. If the user elects to goto the next chapter at 728, but there are no further chapters marked,the recording simply continues playback at 736 until the recording endsor is ended by viewer intervention (at any time). If the viewer attemptsto proceed to a next chapter and there is none, a message can beprovided if desired between 732 and 740 or alternative action such asproceeding to the end can be carried out in alternative implementations.Other variations will occur to those skilled in the art uponconsideration of the present teachings.

The particular analysis carried out at 720 can take many forms dependingon the amount of information conveyed in the score graphic and dependingupon the particular sport. However, in all cases, analysis of the scoregraphic provides a large amount of information that is likely to be ofdirect interest to one wishing to view only highlights. A simple exampleanalysis is presented in FIG. 8 of the analysis carried out to determineif a change in the baseball score graphic is indicative of any of thetypes of plays listed in the example menu 600. In this example analysis,at 802 the change in the score graphic is analyzed at to determine if an“out” count has changed. If so, the filter registers the criterion ofplays resulting in “outs” as being matched at 806. The analysis canproceed whether the result is affirmative or negative at 802 to 810 todetermine if a score changed. (Note that that both an “out” and a scorechange can occur even on the third out in certain circumstances, hencethe full analysis can be conducted in each case to detect specialcircumstances not itemized herein.)

If the score changed at 810, then the filter registers the criterion ofscoring plays as being matched at 814, and control passes from either810 if no match or 814 if matched to 818. If the bases are empty and thescore has incremented by one plus the previous number of runners on baseat 818, then the criterion of a home run is met at 822 and controlpasses to 826 whether a match was made at 818 as indicated at 822 ornot.

At 826, if the batter advances to first base or beyond and/or the scoreadvances this is indicative of a base hit (including a home run) or awalk and the filter match criteria for a base hit/walk is registered at830. Control then passes to 834 from either 830 if a match is made orfrom 826 if not.

At 834, if an inning-count or the top of inning/bottom of inningindicator changes, then the criteria for an inning change or top/bottomchange filter match is met at 838. Control then passes to 842. Theanalysis presented is rudimentary and may be more complex as otherscenarios are examined. Hence, the filter output can either be examinedat 842, or further processing through a more complex decision tree(possibly using the results of prior filter matches) can be carried outat 842. If any match is made to the filter criteria at 850, an output of720 for “yes” is produced, while no matches for 850 will result in a“no” output 720. It will be evident, that the above process is easilymodified to bypass any tests that are not elected as items that theviewer desires to see in menu 600 by simply bypassing that portion ofthe process in accord with flags set when the viewer makes selectionsfrom menu 600.

Marking of score graphic events can be handled in many ways, and themarks can be stored as a part of the audio or video files or part ofmetadata or user data in a separate file without limitations. Severalexamples are presented herein as to how the markings can be situated intime, but those skilled in the art will appreciate that other techniquescan be used within the scope of the present teachings upon considerationof these teachings. The score graphic information is a strong indicatorthat an interesting plays has occurred in the immediate past (e.g.,generally within the last few seconds or possibly one or two minutes).FIG. 9 shows one technique for addressing the time discrepancy betweenthe score graphic event and the time the viewer would likely wish tobegin viewing the unfolding play itself. In this example time line,score graphic events 904 and 908 represent noteworthy score graphicevents respectively. If one correctly assumes that the play of interestoccurred just prior to the time of the score graphic event, one canactually place the chapter mark at times indicated by 912 and 916. Then,during playback the playback process can be simplified to go directly tothe chapter mark and begin play.

However, if the system is devised to intelligently place chapter marks,it can be noted that some events are more likely than others to havetaken place further in the past (prior to the score graphic event). Forexample, in football, one may wish to see several minutes of play(covering a span of possibly several downs) prior to a touchdown, butonly just enough time to capture an actual field goal or point after—saytwenty or thirty seconds. Hence, if an analysis of the type of event iscarried out during registration, storage or analysis of the chaptermarks, the location of the chapter marks relative to the score graphicevent can be customized to more efficiently capture the desiredhighlights.

FIG. 10 depicts a time line for setting a playback offset duringplayback. This offset can either be recorded with the chapter mark orduring analysis and playback. In this case, score graphic events 1004and 1008 correspond in time to the chapter mark, but a playback timeoffset is used to start playback so as to capture the significanthighlight interval. The playback time offsets can again be matched tothe type of event if desired as indicated by playback offset 1 andplayback offset 2 so that playback begins at 1012 or 1016 respectivelyin certain implementations, or a single offset value can be used for allevents. It is noted that the offset can be greater for some types ofchanges in the graphic than for others. An analysis can be performed asdisclosed to determine what type of action has, or is likely to haveoccurred. For example, the unfolding in a baseball game of a grand slamwould likely be expected to take more time than a stolen base, thus theoffset should be bigger. This variability of the offset time can bepreprogrammed based on analysis of historical game events, set by a useraction or ignored and set as a fixed value in various implementationswithout limitation.

In a further example, a viewer interested only in how his favoriteplayers are batting in a baseball game may wish to derive the chaptermarks in such a manner so as to only show intervals of time in which hisfavorite team is batting. Conversely, a viewer only interested in hisfavorite pitcher may only want to take note of the time when hisfavorite team is in the field. Thus for cases in which the score graphicevent involves only an inning or half-inning change, a small or zerooffset might be appropriate. Many other variations will occur to thoseskilled in the art.

The processes previously described can be carried out on a televisionreceiver device (e.g., STB, PVR or similar system) or a service provideras noted below using a device operating with one or more programmedprocessors such as system 1100. In such a system 1100 as depicted inFIG. 11, a computer processor 1104 oversees operation of the processesdescribed herein as well as general control of the television receiverdevice. Computer system 1100's central processor unit (CPU) 1104 has oneor more associated buses 1110 used to connect the central processor unit1104 to memory 1108 such as Random Access Memory and/or Non-VolatileMemory in combination with disc storage in any suitable manner and inany suitable combination. A television receiver/tuner/decoder 1112receives television signals, digital or analog in order to tune anddecode for display on a display 1116 via display interface 1120 andplayback audio via audio circuits 1124.

System 1100 also has access to optical, mechanical (hard disc) or solidstate storage 1128 for storing the recorded games discussed herein aswell as the bookmarks and other bulk data or computer programsincluding, but not limited to, program files and data files. System 1100may also be coupled to a local area network (LAN) and/or wide areanetwork (WAN) and/or the Internet 1132 using a network connection 1136such as an Ethernet adapter. A user interface can be provided by use ofa remote controller 1140 that communicates either one-way or two-waywith the system 1100 via the remote control interface 1144. The viewercan utilize the remote controller 1140 in conjunction with an on-screenuser interface to make menu selections and to “jump” to a time in theplayback associated with the chapter point.

Memory 1108 stores an operating system (not shown) as well asoperational computer programs used in carrying out the functionsdescribed previously. However, it is noted that the functions previouslydescribed can be carried out in hardware equivalent logic devicesspecially designed for this purpose in equivalent embodiments. In thisarrangement, which should not be considered limiting, the memory andstorage 1108 includes a program module for playback control 1150, aprogram module 1154 for carrying out the chaptering process describedpreviously, a pattern matching module 1158 that carries out patternmatching functions (e.g., for matching patterns in section 532 of scoregraphic 500), an optical character reader (OCR) engine 1162 used to readtext within blocks of the score graphic, and event filter logic modules1164 that carry out event filtering functions such as those depicted in720 of FIG. 8. The display interface 1120 incorporates a frame buffermemory 1166 that stores one or more frames of video for presentation todisplay 1116. This frame memory can be accessed for analysis of acurrent display of the score graphic by CPU 1104 using the programmodules depicted in memory 1108.

Those skilled in the art will appreciate that many variations in thearrangement of the program modules and the hardware just described arepossible without departing from embodiments consistent with the presentinvention.

As described above, the receiver device with PVR functions analyzes thevideo to determine score graphic events that are indicative of creationof chapter points. An alternative embodiment is for a service providerto perform an analysis algorithm such as that described herein and todownload the chapter points (event records) to the receiver as a form ofmetadata. The metadata delivered to the receiver could be in the form ofa list of event records associated with a given program title. Eachevent record in the program would provide the time of the event and anevent type. Types of events for an example baseball game could be changein outs, change in innings, change in base runner, change in score, etc.The receiver can take this metadata and create the desired userinterface. Metadata such as this can be sent in real-time as well, andaggregated for later use in the receiver, or can be sent as a batch at atime after the presentation of the programming.

Yet another variation is to provide the receiver a template of the scoregraphic that is pertinent to a particular program. The template can betransmitted from the broadcaster (e.g., as metadata) and received by thereceiver or stored at the receiver in a database of known score graphictemplates, for example. Such a template can provide information to aidin location of score graphic data, such as (for an example baseballscore graphic):

on-screen coordinates where the score graphic and/or elements thereofare located;

coordinates (e.g., for rectangular areas) containing team names;

coordinates (e.g., for rectangular areas) containing half of inning;

coordinates (e.g., for rectangular areas) containing inning number;

coordinates (e.g., for rectangular areas) containing count of balls;

coordinates (e.g., for rectangular areas) containing count of strikes;

coordinates (e.g., for rectangular areas) containing outs; and

coordinates of first, second, and third bases graphic representations.

This method saves the receiver the trouble of analyzing the video andthe graphics to derive this information, but in other implementations,the receiver can find the graphic by looking for displayed elements thathave a geometry that is stationary and has changing elements consistentwith a score graphic. Other score graphics can contain informationrelating to other sports, games and competitions between teams orcompetitors.

Still another variation is to involve the user in the analysis process.A user interface could be designed to ask the user to identify the partsof the score graphic. Each area could be associated with a data type.The user can identify the type as “team name,” or “outs” or “runs” or“balls/strikes,” etc. The analysis in that case would only need to go sofar as to identify the parts. This can be done by an analysis of thedisplay (so called “screen scraping”), where the areas of constant video(the score graphic area) can be analyzed and determine to holdtext/numbers or symbols. The location of the score graphic can bereadily found by comparing many successive video frames to noticeunchanging areas. It is likely that enough pixels will change in onesecond interval to eliminate them from being in the “score graphicarea”, and typically, score graphics are rectangular (or nearly so) inshape. This renders analysis of an image to find the location of a scoregraphic a relatively straightforward video analysis.

Once a score graphic is located and the parts of the score graphic areidentified, the preceding processes can be carried out to facilitate therapid viewing of highlight points in a televised sports program in themanner described. Thus, a method consistent with certain implementationsinvolves at a television receiver device, receiving a television programdepicting a sports program, where data associated with the sportsprogram is represented in a score graphic that is to be displayed on atelevision display overlaying images of the sports program; storing thetelevision program depicting the sports program; analyzing the scoregraphic using optical character recognition and pattern matching toidentify a change in at least one element of data displayed in the scoregraphic from one video frame to a subsequent video frame; storing dataassociated with the time that the change in the element of datadisplayed in the score graphic occurred, where the data includes anactual time of occurrence of the change in the element of data displayedin the score graphic; playing the stored television program; filteringthe data change in the score graphic display according to at least onefilter criterion to determine a category of event associated with datachange in the score graphic display; upon receipt of a signal generatedby a viewer action, jumping to a point in playback time associated withthe mark that occurs at a time prior to a score graphic event time,where the jumping is to a point in time that is offset from the actualscore graphic event time by an offset time, and where the score graphicevent record meets the at least one filter criterion; and where the atleast one filter criterion is selected by a user from a menu of eventsthat the viewer chooses to use as chapter marks.

Another method involves at a television receiver device, receiving atelevision program depicting a sports program, where data associatedwith the sports program is represented in a score graphic that is to bedisplayed on a television display overlaying images of the sportsprogram; storing the television program depicting the sports program;analyzing the score graphic to identify a change in at least one elementof data displayed in the score graphic from one video frame to asubsequent video frame; and storing an event record associated with thetime that the change in the element of data displayed in the scoregraphic occurred.

In certain implementations, the event record represents an actual timeof occurrence of the change in the element of data displayed in thescore graphic. In certain implementations, the chapter mark is derivedbased on a time preceding the time of the event record. In certainimplementations, the analyzing comprises optical character recognitionanalysis. In certain implementations, the analyzing comprises patternmatching analysis. In certain implementations, the method involvesplaying back the stored television program, and upon receipt of a signalgenerated by a viewer action, jumping the playback to a point in timeassociated with a chapter mark. In certain implementations, the jumpinginvolves jumping to a time prior to an event record by a prescribed timeoffset. In certain implementations, analyzing the score graphic involvesdetermining a type of action that occurred in the sports program. Incertain implementations, the prescribed time offset is established on abasis of the type of action determined by the analyzing. In certainimplementations, the method further involves filtering the data changein the score graphic display according to at least one filter criterionto determine a category of event associated with data change in thescore graphic display. In certain implementations, the jumping is to apoint in time associated with a chapter mark that meets the filtercriterion. In certain implementations, the method involves filtering thedata change in the score graphic display according to at least onefilter criterion to determine a category of event associated with datachange in the score graphic display. In certain implementations, the atleast one filter criterion is selected by a user from a menu of eventsthat the viewer may wish to chapter mark. In certain implementations,the method further involves locating the score graphic and identifyingelements of graphics or text data within the score graphic. In certainimplementations, the analyzing the score graphic is carried out duringreception of the television program. In certain implementations, themethod involves locating the score graphic by using a template. Incertain implementations, the template is received by the televisionreceiver device or stored at the television receiver device in adatabase of known score graphic templates.

A computer readable storage device such as a non-transitory storagemedium can be used to store instructions which, when executed on one ormore programmed processors, carry out any of the methods describedherein.

A device consistent with certain implementations has a televisionreceiver receiving a television program depicting a sports program,where data associated with the sports program is represented in a scoregraphic that is to be displayed on a television display overlayingimages of the sports program and a processor. A storage device storesthe television program depicting the sports program under control of theprocessor. The processor analyzes the score graphic to identify a changein at least one element of data displayed in the score graphic from onevideo frame to a subsequent video frame. The processor further storingan event record associated with the time that the change in the elementof data displayed in the score graphic occurred.

In certain implementations, a chapter mark is derived based on a timepreceding the time of the change in the element of data displayed in thescore graphic. In certain implementations, the analyzing comprisesoptical character recognition analysis carried out in the processor. Incertain implementations, the analyzing comprises pattern matchinganalysis. In certain implementations, the processor retrieves and playsback the stored television program, and upon receipt of a signalgenerated by a viewer action via a user interface, jumps the play to apoint in time associated with a chapter mark. In certainimplementations, the jumping involves jumping to a time prior to anevent mark time by a prescribed time offset. In certain implementations,the processor analyzes the score graphic by determining a type of actionthat occurred in the sports program. In certain implementations, theprescribed time offset is established on a basis of the type of actiondetermined by the analyzing. In certain implementations, the processorfilters the data change in the score graphic display according to atleast one filter criterion to determine a category of event associatedwith data change in the score graphic display. In certainimplementations, the jumping is to a point in time associated with achapter mark that meets the filter criterion. In certainimplementations, filtering the data change in the score graphic displayaccording to at least one filter criterion to determine a category ofevent associated with data change in the score graphic display. Incertain implementations, the at least one filter criterion is selectedby a user from a menu of events that the viewer may wish to chaptermark. In certain implementations, the analyzing the score graphic iscarried out during reception of the television program. In certainimplementations, the processor is further programmed to locate the scoregraphic by using a template. In certain implementations, the template isreceived by the television receiver device or stored at the televisionreceiver device in a database of known score graphic templates.

Those skilled in the art will recognize, upon consideration of the aboveteachings, that certain of the above exemplary embodiments are based inpart upon use of one or more programmed processors. However, theinvention is not limited to such exemplary embodiments, since otherembodiments could be implemented using hardware component equivalentssuch as special purpose hardware and/or dedicated processors. Similarly,general purpose computers, microprocessor based computers,micro-controllers, optical computers, analog computers, dedicatedprocessors, application specific circuits and/or dedicated hard wiredlogic may be used to construct alternative equivalent embodiments.

Certain embodiments described herein are or may be implemented using oneor more programmed processors executing programming instructions thatare broadly described above in flow chart form that can be stored on anysuitable electronic or computer readable storage medium. However, thoseskilled in the art will appreciate, upon consideration of the presentteaching, that the processes described above can be implemented in anynumber of variations and in many suitable programming languages withoutdeparting from embodiments of the present invention. For example, theorder of certain operations carried out can often be varied, additionaloperations can be added or operations can be deleted without departingfrom certain embodiments of the invention. Error trapping, time outs,etc. can be added and/or enhanced and variations can be made in userinterface and information presentation without departing from certainembodiments of the present invention. Such variations are contemplatedand considered equivalent.

While certain illustrative embodiments have been described, it isevident that many alternatives, modifications, permutations andvariations will become apparent to those skilled in the art in light ofthe foregoing description.

What is claimed is:
 1. A method, comprising: at a television receiverdevice, receiving a television program depicting a sports program, wheredata associated with the sports program is represented in a scoregraphic that is to be displayed on a television display overlayingimages of the sports program; where the television receiver device istuned to a selected television network, and where a location on thetelevision display where the score graphic appears is determined bymatching the displayed image to one or more template used by theselected television network; storing the television program depictingthe sports program; analyzing the score graphic using optical characterrecognition and pattern matching to identify a change in at least oneelement of data displayed in the score graphic from one video frame to asubsequent video frame; storing data associated with the time that thechange in the element of data displayed in the score graphic occurred,where the display change is identified by optical character recognitionand pattern matching analysis, and where the data includes an actualtime of occurrence of the change in the element of data displayed in thescore graphic where the change in the element of data displayed in thescore graphic is identified by optical character recognition and patternmatching analysis; playing the stored television program; filtering thedata change in the score graphic display according to at least onefilter criterion to determine a category of event associated with datachange in the score graphic display; upon receipt of a signal generatedby a viewer action, jumping to a point in playback time associated withthe mark that occurs at a time prior to a score graphic event time thatis offset from the actual score graphic event time by an offset time,and where the score graphic event record meets the at least one filtercriterion; and where the at least one filter criterion is selected by auser from a menu of events that the viewer chooses to use as chaptermarks.
 2. A method, comprising: at a television receiver device,receiving a television program depicting a sports program, where dataassociated with the sports program is represented in a score graphicthat is to be displayed on a television display overlaying images of thesports program; where the television receiver device is tuned to aselected television network, and where a location on the televisiondisplay where the score graphic appears is determined at least in partby matching the displayed image to one or more templates used by theselected television network; storing the television program depictingthe sports program; analyzing the score graphic to identify a change inat least one element of data displayed in the score graphic from onevideo frame to a subsequent video frame, where the change in the atleast one element of displayed data is identified by optical characterrecognition and pattern matching analysis; storing an event recordassociated with the time that the change in the element of datadisplayed in the score graphic occurred, where the change in the atleast one element of displayed data is identified by optical characterrecognition and pattern matching analysis; and; upon receipt of a signalgenerated by a viewer action, jumping to a point in playback time thatis prior to and offset from the time that the change in the element ofdata displayed in the score graphic occurred by an offset time.
 3. Themethod according to claim 2, where the event record represents an actualtime of occurrence of the change in the element of data displayed in thescore graphic.
 4. The method according to claim 2, where a chapter markis derived based on the offset time proceeding the time of the eventrecord.
 5. The method according to claim 4, further comprising playingback the stored television program, and upon receipt of a signalgenerated by a viewer action, jumping the playback to the chapter mark.6. The method according to claim 5, further comprising filtering thedata change in the score graphic display according to at least onefilter criterion to determine a category of event associated with datachange in the score graphic display.
 7. The method according to claim 6,where the jumping is to a point in time associated with a chapter markthat meets the filter criterion.
 8. The method according to claim 2,where the analyzing comprises optical character recognition analysis. 9.The method according to claim 2, where the analyzing comprises patternmatching analysis.
 10. The method according to claim 2, where analyzingthe score graphic comprises determining a type of action that occurredin the sports program.
 11. The method according to claim 10, where theprescribed time offset is established on a basis of the type of actiondetermined by the analyzing.
 12. The method according to claim 2,further comprising filtering the data change in the score graphicdisplay according to at least one filter criterion to determine acategory of event associated with data change in the score graphicdisplay.
 13. The method according to claim 12, where the at least onefilter criterion is selected by a user from a menu of events that theviewer may wish to chapter mark.
 14. The method according to claim 2,further comprising identifying elements of graphics or text data withinthe score graphic.
 15. The method according to claim 2, where theanalyzing the score graphic is carried out during reception of thetelevision program.
 16. The method according to claim 2, where thetemplate is received by the television receiver device or stored at thetelevision receiver device in a database of known score graphictemplates.
 17. A non-transitory computer readable storage device storinginstructions which, when executed on one or more programmed processors,carry out a method according to claim
 2. 18. A device, comprising: atelevision receiver receiving a television program depicting a sportsprogram, where data associated with the sports program is represented ina score graphic that is to be displayed on a television displayoverlaying images of the sports program, where the television receiverdevice is tuned to a selected television network; a processor; a storagedevice storing the television program depicting the sports program undercontrol of the processor; the processor determining a location on thetelevision display where the score graphic appears is determined atleast in part by matching the displayed image to one or more templatesused by the selected television network; the processor analyzing thescore graphic to identify a change in at least one element of datadisplayed in the score graphic from one video frame to a subsequentvideo frame, where the change in the at least one element of displayeddata is identified by optical character recognition and pattern matchinganalysis; the processor storing an event record associated with the timethat the change in the element of data displayed in the score graphicoccurred, where the change in the at least one element of displayed datais identified by optical character recognition and pattern matchinganalysis; and upon receipt of a signal generated by a viewer action, theprocessor being programmed to jump to a point in playback time that isprior to and offset from the time that the change in the element of datadisplayed in the score graphic occurred by an offset time.
 19. Thedevice according to claim 18, where a chapter mark is derived based onthe time proceeding the time of the change in the element of datadisplayed in the score graphic by the offset time.
 20. The deviceaccording to claim 19, where the jumping comprises jumping to a chaptermark.
 21. The device according to claim 20, where analyzing the scoregraphic comprises determining a type of action that occurred in thesports program.
 22. The device according to claim 21, where theprescribed time offset is established on a basis of the type of actiondetermined by the analyzing.
 23. The device according to claim 18, wherethe analyzing comprises optical character recognition analysis carriedout in the processor.
 24. The device according to claim 18, where theanalyzing comprises pattern matching analysis.
 25. The device accordingto claim 18, further comprising the processor retrieving and playingback the stored television program, and upon receipt of a signalgenerated by a viewer action via a user interface, jumping the play to apoint in time associated with a chapter mark.
 26. The device accordingto claim 25, further comprising filtering the data change in the scoregraphic display according to at least one filter criterion to determinea category of event associated with data change in the score graphicdisplay.
 27. The device according to claim 25, where the jumping is to apoint in time associated with a chapter mark that meets the filtercriterion.
 28. The device according to claim 18, further comprisingfiltering the data change in the score graphic display according to atleast one filter criterion to determine a category of event associatedwith data change in the score graphic display.
 29. The device accordingto claim 28, where the at least one filter criterion is selected by auser from a menu of events that the viewer may wish to chapter mark. 30.The device according to claim 18, where the analyzing the score graphicis carried out during reception of the television program.
 31. Themethod according to claim 18, where the template is received by thetelevision receiver device or stored at the television receiver devicein a database of known score graphic templates.